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VIDEO ARCHITECTURES ~- APPROACHING REAL TIME 
Jeffrey Schier 
Aurora Systems 
185 Berry Street 


Suite 444 
San Francisco, 


SUMMARY 


Overview of video architectures 
that are specialized toward real time 
interaction and response. Issues of 
Analog va. digitel techniques, and 
resolution/speed tradeoffs are 
discuased. Novel systems are 
described that achieve real tine 
performance. 


REAL TIME VIDEO 
Some working definitions of 
Video time-scales are : 


Real Time 

Real time refers to visual 
tools that generate pictures fast 
enough to accurately portray movement 
and inatantaneous interaction of the 
machine with an image. This speed is 
commonly locked to the traramiasion 
or recording Frame rate : 

24 frame/asec. for film, 

29,97 frames/asec. NTSC video, and 

25 Framea/second for European video 
CPAL, SECAM). Sequences of frames at 
rates greater than 15 per aecond, 
give @ reesoneble illusion of 
wmooth movement. 


B) Interactive Time 

Interactive Tine refera to @ 
performance level where actions 
generate visible results which 
perceptually can be connected to 
their stimulus. Echoing a charecter 
to a display, tracking of a cursor, 
and responding to a command are 
operations working in Interactive 
time. Roughly thia speed ia from 
1/15 of a second to 5 seconds. 


C) Animated Time 

Animated Time - Animated time 
will inherit all speeds slower than e@ 
human tolerance for interaction : 
between 5 seconds to 1 day/frame. 
Full movement ia perceptible only in 
its aggregate form, after 
accumulating the entire sequence of 
franes. Time lapse photography is an 
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example of animated time; but frame 

by frame video recorders and optical 
disks are seeing increasing usage in 
computer graphic animation. 
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THE VIDEO SIGNAL 


Achieving reel time video 
involves ’running alongside’ the 
video signal to maintain adequate 
bandwidth. The relation of active 
video time to blanking time, gives 
the proportion of overhead : how much 
time is spent synchronizing veraus 
the amount of time for signal | 
tranamission. The use of two fielda 
displaced 262.5 lines apart to 
achieve interlece, causes images to 
be displaced in time by 1/60 of a 
gecond. Processing of motion and 
vertical features need to account for 
the odd lines acanned in one field, 
with the even linea following 1/60 of 
= gecond later, in the other field. 

Grosa field/frame related changes 
must be completed in the Vertical 
Blanking interval to prevent 
‘fleshing’ the acreen with partially 
complete operationa. Color changes 
and input switching are commonly 
locked to the vertical intervel. 
(Figure 1) 
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Figure 1. NTSC VIDEO FIELD TINENG 
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ANALOG end DIGITAL PROCESSING 

For processing of video images, 
enalog and digital techniques are in 
everyday use. Certain processing 
modules appear frequently in video 
equipment : the video amplifier, 
adder, multiplier, electronic switch, 
and keyer, are some examples. Module 
operation isa governed by numerous 
control inputs. These controls can be 
atatic (fixed) or dynamic - changing 
at speeds up to the video rate. In 
the analog domain the controls are 
voltages and currents, while in the 
digital domain they ere digital 
control codes. 

The Digital components require 
specifying the maximumum clock rate, 
the number of bits of resolution, and 
the number base and arithmetic 
exceptions (gaturating/nonsaturing 
addition etc.). The examples shown 
are synchronous, having a clock to 
line up data marching through the 
modules. The Analog components 
perform ‘satureted’ arithmetic, and 
have bandwidth ’rolloff’ and low 
level noise which limits their 
resolution. 


PROCESSING MODULES 

Video Anplifier - 
Analog : Hi-freq OP AMP 
Digital : Multiplier 


Voltage Ceontrplled Video Amp - 
Analog : Video rete multiplier/ 
Programmable gain amp 
Digitel : Multiplier with second 
input as control 


Comparator - High Gain Video Amp 
operating in differentiel mode. 
It achieves a two-state output ; 
a ’1’ if IN_1 > IN_2 or a ’O% 
if IN_2 < IN_1. Useful in flash 


A/D converters and KEY generation. 


Electronic Switeh - one of many 
inputs ere routed to ea aingle output 
by a control signal. <A Video rate 
awitch is a ‘Hard Edge Keyer’, with 
the control input being the KEY. 
(Figure 2) 
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Figure 2. HARD EDGE KEY ER 


Delay Element - 

Analog : Cable Delay, Amplifier 
delay, tapped delay line, quartz 
delay line, Video tape delay line. 

Digital : reclocking register, shift 
register (arbitrary length? FIFO, 

Line buffer, Field buffer, Frame 

buffer, Magnetic Video Diak Optical 

Video disk , Laser Video Disks. 


Adder - Provides the sum of two or 
more inputs 
Analog : video Op-Amp mee 


Digital : two input adder 


Boclean Logic - uniquely digital 
functions : OR, AND, EXOR and 
Complement 


“5 ALU ( Arithmetic Logic Unit) 


Digital : proceasea the additive 
and Boolean functiona 
Priority Encoder : 
Digital : Ouput * binary encoded 
number of the greateat numeric 
input 


Multiplier : Output ia the product 
of the two inputs 


Memory Elements - 
Analog : Hybrid CCD delay line, 
CCD image sengsor/nemnory 
Digitel: Lookup tables, Frene/field 


buffer 


Ancillery Processing Componente 


Phase Locked Loops : aligns the clock 
phase of internal clocks to external 
async. Used in gen-lock (Syne 
Generator lock) to match the external 
timing to internal sddresas/timing. 


A/D converter ~- converts analog 
voltage levels to digital codes 


D/A converter - converts digital 
video to analog outputs 


DC restoration - Analog correction of 
AC’ coupling in video 


Color Encodera / Color Decoders - 
converts R,G,B signals to/from a 
composite form for transmission or 
recording. 
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DIGITAL VIDEO AMPLIFIER 

The digtital video amplifier is 
a 2 input Multiplier, with Input_l 
the Video-In, and Input_2 = Gain. The 
multiplier ia either an LSI circuit 
commonly (8 to 16 bits), or composed 
from programmable memories (PROMS, 
EPROMS) with fixed gaina. (Figure 3). 
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Figure 3: TORS hArLi stan 


THE UBIQUITOUS LOOKUP TABLE 

Unique to digital video 
processing 1a the extensive usage of 
mMenory as & proceasing element. The 
ability to delay, atore, retrieve 
data, and generate functions are 
major applications of memory 
components. One example ia the Lookup 
table or Mapping memory. The Lookup 
table ia a faat clocked memory which 
translatea data on input, to new data 
on its’ output : Output 3 F (Input). 
A fixed table is ‘hardwired’ or 
programmed to one set of values, 
while Read/write lookup tables allow 
programming a variety of functions. 
While the most general of processing 
elements, ita limitations stem from 
the number of input bita and output 
bits which can be placed in one 
memory array. The greater the number 
of entries Caddresses), the larger 
the memory: the longer it takea to 
load or modify. Some contend that 
the frame buffer ia a large lookup 
table with 2 inputs : en X~-Display 
addreaa and a Y-Display addresa. For 
e 512H by 5S12V size frame buffer this 
needs 256K entriea, not conveniently 
loaded in one Video blanking interval 
(1.3 milliseconds). 


Color Lookup Table (Pseudo-Color) 
Given a digital video data 
atream, the input data often 4-12 
bits ‘deep’ is transleted through a 
eet of 3 read/write lookup tables. 
Each table is asaigned to an output 
channel: one to red, one to green and 
one to blue. The output of the 
tables are routed to Video speed 
Digtial to Analog Converters (D/A’a) 
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and sent to an R,G,B monitor or NTSC 
color encoder (Figure 4). The tablea 
set the correspondence between the 
nonochrome input and colors at the 2 
output. If all 3 tables are set to 7 
unity (a ramp), the monochrome input 
generates a monochrome output. If 
specific entries are set to different 
values in each table, a color will be 
output when the table entry showa up 
on the video streaar. 
If the input to the table conea 
from a frame buffer scanned out at - 
display rate, the tables connect ~. 
color to frame buffer data. In 
computer graphic Paint systema, 
operations are perfromed on the frame 
buffer with a corresponding color 
map, tagging color to pixel data. 





An interesting effect used in 
paint systems ia the ‘rotation’ of 
the color map entries. The entries 
are bubbled up through the table by 
copying the entries from locations 
below to one location above. The top 
entry ia move to the bottom of this 
‘Map ribbon’. With «a sequential grey 
acale fluid movement, and Marquia 
effects simuleted. This operation 
nuat be performed at frame rate, in 
the vertical blanking intervel to 
prevent ’fleshing’ the acreen while 
changing the table entries. 


Remapping Memory 


Another use is to map input to 
output through a desired function. 
The input marches through the memory 
and arrives at the output one clock 
later, mapped through 4 new function. 

This function can be exponential 
linear, clustered numbers, or 
singular entries. (Figure 9) 
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Figure 5. REMAPPING MEMORY 


Gamma Correction ~ Fixed functions 
are used as ganaa correction tables 
placed before output D/A converters 
to correct for the ‘intentional’ 
non-linear response of video monitors. 


Thresholding - By turning apecific 
entries in the teble OFF, with the 
remainder set to & grayscale, only 
designated gray values make their way 
through the teble. This 1&4 useful to 
view restrained section of the input 
grayscale, or to maak out unwanted 
gray values. 


Intensity Compensation - If an input 
wignal/image hes 4 nonuniform 
grayacale response caused by 
incorrect exposure or brightness 
offeet, the table is set to the 
inverse function, effectively 
cancelling out the input error. 


Selector - since the table mapa 
Addreas to Date, certain address bits 
can be masked off through the table, 
@liminating them from the mapping 
function. Thia elininates the 
contribution of groups of gray 
levels, ’Maaking’ or turning them 
off. If the input is from the frame 
buffer, bit plene masking can be 
achieved in the output lookup table. 


Control Table - As a control table, 
inatead of greyscale values being 
mapped into other grayscale values; 
the output of the table is used to 
control other hardware. Thia control 
infornetion corresponds to the 
grayecale value of each pixel, 
allowing each pixel gray value to 
have an independent function assigned 
to it. Control of input selectors, 
ALU functions, and deatination 
controla are epplicationsa for control 
tables. (Figure 6) 
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Figure 6. COMTROL TABLE 


FRAME BUFFER 

The frame buffer is a large block 
of memory applied to storage of the 
entire video frame. When the memory 
dja time locked to the horizontel and 
vertical sync, the output pixels form 
a stable pattern for display. 
Allocation of memory cycles, 
input/output data and address 
selection are essential to achieve 
full speed video rates. Time must be 
allocated for ‘refresh’ when using 
dynamic memoriea to maintain data 
integrity. ¥ 

The number of inputs and outputs’ 
are the PORTS of the frame buffer. 
For a given memory bandwidth memory, 
the fight for memory bandwidth by 
each port isa a major concern of 
system performance. For 4 display 
related frame buffer one Read Port is 
dedicated to the screen access. T° 
ship information into the buffer a 
Write Port muat be dedicated. 


a 


Single Cycle Memory 


In a single cycle memory system ¢ 
the memory bendwidth can only sustain 
one access per pixei), the write 
function can only occur while Not 
reading. To prevent interruption of 
the screen diaplay ‘(resulting in the 
diatesteful flashing or glitching of 
pixels), access must be limited to 
the video blanking intervals. 


Video Digitizing end Read/Write Access 
Curiously the need for 
bidirectional access to the frane 
buffer ia often ignored. Early frane 
buffera for computer graphics 
left out provisions to ’READ BACK’ 
the images just written. 
The data went in, reached the display 
but could not be extracted as pixel 
data by the computer port. Similar 
ommisions are found in Lookup tables 
that are Write-Only. This ommission 
was provoked by building *Diaplay-Only’ 
devicea, that generated line drawings 
and boxea. With greyscale inage 
generation and retrievel, the Read 
and Write port should alweys be 
present. What goes in aust come out. 
The other side moat often 
ignored in frame buffers is video 
input capability. If the memory can 
be scanned out at video rate, it can 
aleo be ’acanned in’ in real tine 
(video digitizing). In 4 single port 
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ayatem the output can ’echo’ the 
input video giving an output while 
digitizing, useful for adjusting or 
focusing the input source. The video 
input is received from the video rate 
A/D converter. To digitize real time 
video, the display system MUST 
GEN-LOCK, to elign its internal 
timing to the external video source. 
(Figure 7) 





Figure 7. Video Digitizing 


In a read/modify write frame 
buffer, the image pops out one frame 
later than it’s input video. This 
delay ia useful in image 
accumulation, color noise reduction, 
edn other frame comparison 
operations. A minimum set of ports 
for @ real time video buffer are :; 


1) A real time video input Port 
(Write Port) 

2) A real time video output Port 
(Read Port) 

3) A computer Write Port and Readback 
Port 


FEEDBACK PROCESSING 

Given a delay element and a4 
processing element, feedback 
processing is possible. A portion of 
the output data is rerouted and 
combined with the input. This 
technique ia employed in recursive 
digital filters; and with the delay 
set to one frame - time and motion 
effects can be processed. 


Configuration ~- 


Video_In routes to one input of an 
ALU, output of the ALU goes to 
digitizing input of the frame buffer, 
output of frame buffer runs to the 
second input of ALU. The input is 
compared against the output, with the 
reault of the comparison selacting 
the ALU function. Variationa on thia 
configuration perform :° 


1) Frame Accumulation - 


(Noise Reduction, Frame averaging) 


The input ia added to the previous 
frane, scaled by one half, then 
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rewritten to the frame buffer. Thia 
is run for N frames. Stationary low 
light level images will be 
accumulated into the frame buffer. I£ 
the feedback gain and input gain are 
adjuated this configuration will 
perform noise reduction. (Figure 8) 





Figure 8. - Frame Acoumuiate 


2) Motion Detection - 

(Motion Detection, Noise 
accumulation) The control selection 
isa set to two ALU functions. A 
comparator is used to select the ALU 
function, based on the difference 
between the current input and the 
previous frame. This writes only 
changing information back into the 
frame buffer, if the input video 
exceeds a certain THRESHOLD. Levels 
below the threshold are considered 
noise and are not accumulated. 
(Figure 9) 





Figure 9 7~ Motion Detectian 


SCAN PROCESSOR 

A unique real time video 
processor is the scan processor. Its 
principle of operation is to 
intercept the aweep signals of a 
diaplay monitor and modulate these 
signals with control voltages. An 
Analog example is the Rutt/Etre scan 
processor, with a simplified block 
diagram shown in Fig. 10. The raater 
ia manipulated by control voltages 
feeding two chains of four-quadrent 
multipliera and a summing amplifier, 
placed between the H and V ramp 
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generators, and their corresponding 
deflection yokes. The video signal 
runs through a multiplier followed by 
a asaumming amp for intensity and 
brightness control. 

The control voltagea ere 
driven either from ’static’ voltage 
sources or from Function Generators 
locked to : Horizontal sync, Vertical 
async, or themselves (’freerunning’). 
AM and FM control allow cascading 
these control signals. Images sre 
‘rescanned’ by 4 camera facing the 
modulated monitor, with optical 
effects achieved by placing objects 
between the resacan camera and the 
monitor. 


The raester’s size, position 
and intena@ity can be modulated 
through voltage controi signala. The 
ability to modify the underlying scan 
process along with the video aignal 
are unique properties of the acan 
processor. 

The need for intensity 
compensation of amall rasters, 
resolution loas due to the rescan 
process, and the difficulty of 
achieving repeatable raster movement 
using Analog control generatora have 
been some of the shortcominga of thia 
scan processor. Digital control : 
Signals run through D/A converters,’ 
can gimplify the control structure 
and improve repeatability. 
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Figure 1@. RUIT‘ETRA SCAN PROCESGOR 
S{MPLIFIED BLOCK DIAGRAM 
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PROCESSING AMPLIFIER 

Composite proceasing of the video 
Signal is exemplified by the aptly 
named Video Processing Amplifier (or 
PROC AMP>. The Proc Amp’s primary 
function is to cleanse the compoaite 
Video signal of aberrations injected 
after routing through 4 long chain of 
video components. The Proc Amp 
Beperates the Luminance, Chrominance, 
and Sync information; only to place 
them back together with independent 
control of their levels. The 
luminance brigthnesas (Black Level), 
its contrast (Luminance Gain). Detail 
(Edge enhancement), White and Black 
Clipping (to inhibit ovarshoct/ 
underashoot) are adjustable through 
Voltages driven from front panel 
controls. The chroma infromation is 
extracted (by ita 3.579545 MHZ 
frequency) and the Saturation (GAIN)>) 
and Hue (Phase ) are adjustable. The 
a@ync signal is turther ’Regenerated’ 
to reconatruct any vertical or 
horizontal async diatortions, with a4 
new color Burat aignal inserted on 
the back porch for the chroma 
reference phase. A reconstructed 
video gaignal ia built up through 
control of its ‘component’ parta. 

The Proc amp while designed 
for video correction can also become 
the active component of a video 
effect unit. Substituting front 
panel control voltages, with video 
rate control voltages, the Colorizer 
ia born. The Colorizer’s basic 
function ia the insertion of color 
onto a monochrome signal. The video 
®ignal thua emerges as both an image 
source and a control source tor 
selecting color. The generation of 
the video image, directly from 
Signals or voltages is termed ‘Direct 
Video Synthesis’. Stephen Beck, Dan 
Sandin, Dave Jonea, Bill Hearn, Eric 
Siegel, and George Brown are have 
each developed video processors, 
whose images are generated and 
eentrolled from video and non-video 
rate signala. 


COLORIZER 

The majority of colorizers are 
expangaions upon the Proc Amp with the 
controls derived from the luminance 
Bignal. The diatinction between a 
composite black and white signal and 
a composite NTSC color asaignal is the 
presence of the 3.579545 MHZ 
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g2ubcarrier and ite reference Burst. 
Given & black and white signal, the 
burst and subcerrier is aynthetically 
generated, converting it into a color 
Signal. The method of asubcarrier 
yeneration distinguishes different 
breeds ot colorizers. Three 
diatinetions will be made - the 
modulation based colorizer the 
*alicing’ threshold based colorizer, 
and the NTSC encoder/colorizer. 


Direct Subcarrier Modulation Colorizger 

An example of a modulation’ 
colorizer is the Eric Siegel 
Colorizer shown in block form in Fig 
11. A monochrome signal is input, 
filtered of extraneous 3.58 MHZ 
components, its luminance component 
18 “detail enhanced’ then run to a 
chroma Phase SHift Modulator. This 
modulator links the contrast 
component of the input to the output 
Hue . This phase shifter, is a two 
stage circuit enabling greater than 
360 degree rotations in hue space 
from @ black to white video 
excursion. The amount of phase ahift 
and ita polarity ia seiected through 
front panel controls, as well as 
Starting phase and saturation. Thie 
device is representative of the 
Direct Subcarrier Modulation 
colorizer. 
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Fig. If - Erica Siegel Colorizer 
Simplified Blook Diagran 
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Luminance Classifying Colorizer 

The Threshold based colorizer, 
Classifies luminance into multiple 
bands, with independent contrel 
within each gray region. Two exanples 
of this are the Bill Hearn EAB 
colorizer, and the David Jones 
multiband colorizer. 

The luminance component is 
filtered then shipped to a bank of 
comparators. Each comparator detects 
@® luminance threshold and combines to 
form “bands’ of gray, which gate on a4 
get of chroma/luma controls. If the 
thresholde are non-overlappping, the 
controls operate independently to set 
brighness, saturation, hue, and 
Contrast; in each gray band. 


NTSC Encoder / Colorizer 

This colorizer is based 
around the RGB to NTSC color encoder, 
as the active color encoding element. 
The R,G,B camere inputs are exchanged 
with other control signals, some of 
them at video rates. The use of the 
NTSC encoder reatrains the output, to 
be within the “bandwidth restrictiona 
Of NTSC. Correct observation of 
these limitations helps avoid ‘color 
eamear’, and loat detail in the 
encoded color picture. If the 3 
inputs are from e digital video 
source we arrive at the Paeudo Color 
colorizer. 


Digital Pseudo Color Colorizer - 


The number of classfying 
bands hes been limited practically 
to 10 analog banda. Beyond this, the 
number of control pointa becomes 
inordinately large and unwieldy to 
control. Intereatingly ‘Flesh A/D 
converters’ consiat of the same 
Classifying arrangement : a bank of 
comparators compare the input video 
egainst a chain of voltage 
references. The references are 
equally apaced, end not independently 
adjuatable. The A/D further converta 
the detected levels into an encoded 
binary value before output. This 
*digitized’ video aignal ia passed 
through to a triple set of lookup 
tablea and D/A converters, then to 
the RGB to NTSC color encoder. This 
digitel method simulates many of the 
linear functiona of the analog 
colorizer . 


Schier/ McArthur/ Vasulkea 
Image Processor 

This digital video processor 
contains a vertical interval control 
bus, ygen-lock timing, and a 
microcomputer to orchestrate field by 
field control of the image. The 
digital video paths ere connected to 
the processing modules through front 
panel patching permitting a variety of 
interconnections achemes. The 
components parta& are : 


1) A DEC LSI-11 microcomputer 
coordinating control words for - 
processing and user interface 
functiona. 


«) A Vertical Interval Control Buffer 
LST-11 Interface Control info. is 

loaded into the control buffer during 
the active field, for transfer to 
processing modules during the next 
vertical blanking interval. An 
interrupt is generated to the LSI~11 
after the control buffer ia 
transferred. 


3) A Gen-Lockable Syne Generator - 
timing ia baged upon 3512 H by 486 V 
active screen coordinates. Both video 
sync and Horizontal / Vertical timing 
ig available on the control bus, for 
pickot?® by modulea. 


Processing Modules : 


A) Video Rate Analog to Digital 
Converters (A/D) 


B>) Selectora ~- 3 groups of selectora - 
choosing between 8 horizontal, and 8 
vertical frame locked patterns, and an 
External digital video source. The 
selectors allow bit-wise selection of 
horizontal, vertical and input 
components. 


C) Arithmetic Logic Unita (ALU’s) - 
Combinesw two digital input streams 
into e single output through 
combinations of arithmetic and Boolean 
logic functions. The Boolean 
functions of ’AND’, ’OR’, 

“EXOR’, ’EXNOR’, Ones Complement are 
present. The arithmetic functions ’A 
PLUS B PLUS CARRY’, ’A MINUS B PLUS 
CARRY’, and 2’s Complement are also 
available. Certain combination of 
erithmetic with logical operations are 
possible, with a ’Conatant’ available 
on the ’°B’ input, useful for bit 
masking. 
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D) Lookup Table - an R,G,B lookup 
table with common digital ‘Address 
input’ is present, to perform 
intensity/pseudo color tranaformations 


E) Window generator - three Window 
generatore, form 4n adjustable frame 
for gating/routing the digital 
sources. The ‘Window Frames’ are 
programmable on a pixel/line basis. 
Wipe patterns and title boundaries are 
formed here. 


F) Digital to Analog Convertera (D/A) 
-~ one apiece for red, green and blue 
channels. 


G) RGB to NTSC color encoder - the 
final output is derived from the Reda, 
green and blue components then 
converted to composite NTSC. 


Input is received from camera 
sources, video tape or from the 
internal pattern generator (H and V 
tCimed bar patterns). Camera/VTR 
sources firat go through the A/D 
converters, are front panel patched to 
the processing modules, route to the 
D/A’a for conversion to Red green, and 
blue. The signala then hop into the 
RGB to NTSC encoder, converting to a 
composite NTSC video output, for 
display or further post-processing. 


eH, 


~ So kde pulse 
eeacron 


tAEEAciad PlgchoRheRGA" 


BIBLIOGRAPHY 


cy 
pea [bEO 


SENERA TOR 


Principles of Interactive Computer Graphics Newman and Sproull 


2nd Edition ¢i$879 : McGraw Hill, 


Inc. 


Fundamentela of Interactive Computer Graphics 
Foley and Van Dam c1982 ;: Addison-Wesley Publishing Co., Inc. 


Televison Broadcasting - Equipment, Systema, Operating Fundementals 


Harold E. Ennes c¢ 1979: Howard W. 


Television Engineering 


Donald G. Fink c1952, 2nd Edition, 


Sama and Co. Inc. 


McGraw Hill, Inc. 


Tranamiasasion and Display of Pictorial Information 
D.E. Pearaon , c1975 Haleatead Preas / John Wiley and Sons Inc. 


U.S Color Televigion Fundementala 
D.H. Pritchard, SMPTE Journal Nov. 


A review 
1977, Volume 86 


Tektronix - “Televison Operationsel Measurements - Video and RF for 
NTSC Systema" c1984 3rd edition AX-3323-1 


EIA RS170 (1957), RS-330 (1965), 
RS170A ‘tentative Nov. 


9/4 


RS-343A (1969) 


1977) 





Vasulka Corporation 
25/7 Franklin Street 
Buffalo, N.Y. 

14202 
(716) 856 - 3385 
Submitted by Jeff Schier 


Dear Purchasing, 


Below is a list of Packaging panels, and wire wrap accessories: 


Mupac — 


uantit Catalog 500 Number Description Unit Price Total Price 
1 3233080-01 Single width card extender $ 129.25 
3243080-01 Double width card extender S 199.00 
3312549-03-02 "Mixer" Horizontal card rack 


4/7 Panel rack, 12 connector 
Receptacles, Retma end plates, 
accomodates -— two 326 panels 


two 324 panels, two 323 panels $ 377.25 
2 3264881-01 Triple width Universal panel | 
without socket terminals $109.00 S$ 218.00 
1 3245081-04 - Double width "Sponge" panel 
without socket terminals S 75.00 
1 3244881-01 Double width Universal panel 
without socket terminals $ 75.00 
2 3234881-01 Single width Universal panel 
without socket terminals $53.00 $ 106.00 
7 1301422-01 Socket terminals(100 pieces) $11.90 S$ 83.30 
1 1301402-01 Socket terminals(100 pieces) 
(slotted) $ 12.10 
1 3752530-01 Socket terminal Insertion/extraction 
; tool $ 30.00 
3611600-36 36 position receptacle (10 pieces) S$ 12.00 
3611600-18 18 position receptacle (10 pieces) S$ 7.00 
1311517-01 Mini terminal receptacle $15.40 S$ 46.20 


(100 piecs) 
1 3752503-01 105CFM Fan S$ 39.95 


Mupac Total $1410.05 
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Purchase Number 


Image Emulsifier 
Ordering Information 


October 1978 


a ee re er eee ee erm yer ee eet er ce ee rerey me me rm nh eg ge il a SD ee i Se ee ere ee ee 


B12 


B13 


B14 


B15 


B16 


B17 


B18 


B19 


B20 


B21 


B22 


B23 
B24 


B25 


B26 


B2/ 


B28 


Description Cost 
Summit Distributors -— EMI Filter S$ 19.60 
and Miscillany 
C.K. Wall Sockets and Socket $546.75 
Terminals 
Summit Distributors - Ceramic S$ 30.00 
Capacitors 
Summit Distributors - Capacitors S 54.00 
Unused 
Mupac —- Rack and wire/wrap boards$967.60 
Samtek -— Sockets and Socket terminals 
$370.50 
Summit Distributors - Semiconductors 
S$ 30.00 
Schweber Electronics - $175.75 
Semiconductors 
Hamilton Avnet - Assorted $278.64 
Semiconductors | 
Summit Distributors - Solder S 29.20 
tail sockets 
Unused 
Hamilton/Avnet - Semiconductors, S$ 57.80 
data selectors 
Hamilton/Avnet - Semiconductors, $ 25.95 
assorted 
Summit Distributors - oS 3Se25 
Semiconductors 
Summit Distributors - Interface S$ 16.11 
Semiconductors 
Cramer Rochester — Schottky Ics $101.26 
Page Total S 2756.41 
Total S 7651.96 


Received 


Received 


Received 


Received 


RECEIVEP 


Received 


Received 


Received 


Received 


Received 


Received 
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Ordering — image Pnulsifier 


August 8th, 1978 


Jeffy Schier 





Purchase Order # Description/Company Cost Received? 
- B-O1 Digital Fquiptment Corporation $ 176.25 Yes 
LSI-11 Interface ~— DCK11l~ac 
- B-O2 Powermate SW -5-G $ 265.00 Yes 
Power Supply 
- B= 03 OK Wire and Tools $ 18574 Yes 
Wire Wrap tools 
~ B- OL Mupac ~ Packaging and $1390.30 Yes 
Wire Wrap panels 
- B-05 Harvey Federal (Rochester) $1554.30 Yes 
Buffer memories (Intel P2141-2) 
B = 06 Hamilton Avnet $ 384.610 
Assorted Semiconductors 
~ B=-0O7 Rochester Radio $ 240.60 Yes 
Semiconductors /Memories 
_ B-0o8 Data Delay Devices $ 52.00 Yes 
Delay Lines 
B — 09 Hamilton Avnet 
Control Store Memory (931422) $ 136.80 
lied Unused 
. B 11 Schweber Electronics $ 210.16 Yes 
Arithmetic Logic Units 
Total 8/8/78 $4895055 
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Toni VASULK-=: 
257 FRANKLIN S~-. 
BUFFALO.N.Y. . 22°32 


716-856-3385 


February 10, 1976 


.. Equipment Corporation 
vinton Hills 
: Park - Fairport 
Rochester, New York 14450 
act: Paul Gizzi 
Dear Sirs: 


[ am interested in purchasing the following equipment from you: 


1 RXV 11 - BA $3900.00 
i RXO1K - 10 75.00 


Tax 72% 278.25 


Total $4253.25 


Would you kindly notify me as to what billing procedure will be necessary’ 
Thank you for your kind attention to this matter. 


Sincerely» 


Pctusfaul fad ka (wat) 


Bohuslav Vasulka 


5V /msd 








DIGITAL IMAGE PROCESSING 





Thomas W. Moxon 

EE 499 - Work Study Program 
Professors: 

Woody Vasulka 

David Bennenson 








The subject of my independent study this semester has 


been the theory and methods of digital image processing. Digital image 
processing is the term applied to the numeric codification and subsequent 
algorithmic manipulation of pictorial information. 

This codification, or quatization, of the image is 
performed on a point by point basis as the picture is scanned, where 
each point is assigned a number corresponding to the luminance value of 
the picture at htat point. The range of numbers that may be assigned to 
each point determine the amount of discrete intensity changes that may 
be detected; while the number of individual points determine the smallest 
area of the picture that may be encoded. This quantization is normally 
implemented with a television camera and analog - to - digital conversion 
circuitry; where the digital word size corresponds to the range of numbers 
which may be assigned, and the analog sampling rate determines the number 
of individual points that may be resolved from the original picture. 
I will expand on these concepts in the section on quantizers. 

The digital processing of the-scene content however, 
tends to follow two different paths, that of pipeline processing and 
that of storage, or frame buffer processing. Pipeline processing is, 
as the name implies, a method where an operation is performed upon the 
scanned stream of picture elements as they are passed to the output 
display or recording unit, in a continuous fashion. The main advantage 
of this method is the high speed with which operations may be performed, 
allowing the capacity for real-time motion within the frame. However, 
since the frame exists only as a continuous stream of picture elements, 
hereafter refered to as pixels, there is no capability for operations 


which require knowledge of, or comparision with, past scene information. 








Examples of operations that may be performed with pipeline processing are 

color mapping, intensity remapping, window formation and digital keying. 
Frame buffer processing is the term applied to the 

storage of single frames, or sequences of frames; allowing transformations 

of pixel location within a frame, or over a number of frames. This method 

gives the capability of performing operations which are time variant, 

such as compression and expansion of the image, trajectory tracing, and 

pattern recognition. It's main disadvantage though, is the extremely 

slow speed of storage and recall, resulting from the sheer volume of 

data involved. Most image processing systems encorporate both methods, 

under computer control, to achieve a wider variety of operations. 

The "Image Emulsifier" , which I have been helping the Vasulka Corp. 


to develop this semester, is just such a system. 


Description of the Image Emulsifier 


The Image Emulsifier system is a microprogrammable 
processor for the aquisition,storage,modification,and distribution of 
digitally encoded images of four bit resolution. The system currently 


consists of the following subsections: 


]) Quantizers- Two high-speed analog- to ~-digital converters operating on 
separate buses are employed for the real-time aquisition of two camera 
images by the system. I constructed and tested the quantizers, which are 


based around the new Advanced Micro Devices 6688 integrated circuit. 





2) Image Buffers- Eight image buffers of 8k deep by four bits wide with 
high-speed access (200 nanosecond) time. Entering each buffer are 
four busses carring control signals,address and data information. At 
any time, any buffer may be connected to any one of the four busses. 
This allows for expansion of the image resolution by concatenation of 
several buffers onto the same bus; for example, connecting two buffers to 
the "A" bus makes them appear to the bus as a widened, 8K by 8 bit buffer. 
Connection of a particular buffer to a bus is initiated by the buffer 
prtority logic, associated with each buffer. I constructed four of the 


eight image buffers, and am in the process of testing them at this time. 


3) Microprogrammed Microprocessor-— A high-speed (200 nanosecond) 
microprocessor is connected to two of the four busses fed to the image 
buffers. By controling two busses, reading or writing may be done to 
different locations in two different buffers. This gives the capability 
of variuous picture transformations, such as picture inversion, compression, 

| , an? 
expansion, edge extraction, and outlining. A program control store 
and sequencer gives instructions to the microprocessor, which is structured 
around an 80 bit pipeline. This control store has the capability for 
expansion which will allow Prommed, or hardware stored programs to be 
permenently resident in the memory, to facilitate execution of the 
most commonly used microprograms. The microprocessor is based on the 
Advanced Micro Devices 2900 series of integrated circuits, which employ 


the latest method of microprocessor design, that of the Bit Slice concept. 





4) Microcomputer Interface- The particular interface involves connection 





of the Image Emulsifier to an LSI~ll microcomputer. The interface connects 
the LSI-11 to the microprocessor and to one bus of the buffers. A buffer 
request register is on the interface, to allow requests for usage of a 
particular buffer. To access the data of a buffer, the LSI-11 must 

first request usage; if it is the highest priority device, a grant 
register will indicate that it has usage of the buffer. This grant 
register can be enabled to generate interrupts, upon getting the butfer 
that was requested. Once a request/grant sequence is successful, a IK 
block of data may be written to, or read from, the granted buffer. To 
write to other sections of the buffer a memory page register is used, 

to map the 1K memory space into one of the pages of the 8K buffer. 

The LSI-11 is also responsible for setting the buffer priority 
registers on the eight image buffers. The LSI-11 must monitor the buffer 
activity, to time the change of priorities which affects the next memory 
cycle of the buffers. The LSI-ll is also responsible for loading the 
contro] store of the microprocessor, and has various control registers 
to facilitate this operation. It controls the run/single step/stop of 
the microprocessor to allow debugging of the microprograms, and serves 
as the main memory for controling programs. At the present time we are 
implementing an assembler/disassembler to ease the task of microprogramming 


the Image Emulsifier. 





6 a yee 


5) Buffer and Screen Addressing Circuitry- This section accommodates 





the generation of the horizontal and vertical timing to scan out, 

or write in, digitized video information to the image buffers. This will 
allow such effects as adjustment of the height and width of the 

picture, shifts of the position on the screen, and various pixel 
position remappings, 

A preliminary constraint is to lock the phase of the high frequency 
clock to the video subcarrier phase, to avoid color shifts in the system. 
The timing of the X-Y timing chain , determining the pixel coorinates, is in 
turn locked to this high frequency clock. The high frequency clock itself 
sets the 'time grid' which specifies the maximum and minimum resolution 
attainable in the scanning of the buffers. With a high frequency clock of 
9.75524 MHz and an active video line of 52.4 microsecond duration, this 
forms a total number of 512 picture elements as the maximum resolution of 
the system. However, due to the buffer cycle time of 200 nanoseconds the 
resolution is further limited to 256 pixels as the maximum. Thus,to store 
a 256 by 256 image requires some 65,536 memory locations, which is just the 
size of our complete buffer memory. By using all eight buffers simultainously 
one frame of 256 by 256 resolution may be stored, but more frames may be 
stored simultainously if a lower resolution is used. For instance, if 
128 by 128 resolution is used. then 4 frames may be stored at one time, and 
if 64 by 64 resolution is used then 8 frames may be stored. A control 
register is employed to indicate the desired resolution mode of the buffers, 
which adjusts the timing logic accordingly. 

A mapping memory of 256 words by 8 bits is placed on the output 
of the X-Y address timing , one for the X and one for the Y addressing. 


These memories may be loaded with a function, which will translate the 





sequential X and Y into a different addressing scheme. These memories 
can be loaded during the inactive scan and utilized during the next active 
scan. 

The use of high speed multipliers on the X-Y signals is 
another possiblity for image transformations. A 200 nanosecond 
multiplier of 16 bits width is employed on one bus to facilitate the 
loading of maps with transformations, or to use directly on the image 
data itself. In instances where transformations require pixels to be 
Skipped or discarded, such as image compression, one place$the task 
of which pixels to be skipped upon the round off error of the multiplier 
itself. 

For more random addressing or data generation, a pseudo-random 
number generator has been implemented, with a variation on the shift 
register that allows the next input to be determined from the addition 
of selected bits of the register. This gives repeating sequences of 
programable length. However, by loading the register directly from the 
X-Y clock an unlocked noise source is then generated, due to the mismatch 
of the pseudo-random sequence to the video line/field blanking rate. 

My role in this section of the system has been the design and 
construction of the phase locked loop for the high frequency clock, 
and testing of the address counters and maps. The multiplier circuit 


remains to be wired and tested, possibly this week. 


6) Output Selection Logic- The output selection logic is responsible for 
routing the buffer outputs and other units to one or more of the four 
outputs of the system. A screen addressing unit further segments the 


outputs into four quadrants, according to the cartesian coordinate system. 
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[ me subject of my independent study this semester has 


been the theory and methods of digital image processing.| Digital image 
processing is the term applied to the numeric codification and subsequent 
algorithmic manipulation of pictorial information. 

This codification, or quatization, of the image is 
performed on a point by point basis as the picture is scanned, where 
each point is assigned a number corresponding to the luminance value of 
the picture at htat point. The range of numbers that may be assigned to 
each point determine the amount of discrete intensity changes that may 
be detected; while the number of individual points determine the smallest 
area of the picture that may be encoded. [nis quantization is normally 
implemented with a television camera and analog - to -— digital conversion 
circuitry; where the digital word size corresponds to the range of numbers 
which may be assigned, and the analog sampling rate determines the number 
of individual points that may be resolved from the original picture. 


I will expand on these concepts in the.section on quantizers. | 


4 


ii rem 
Pnee eae The digital processing of theescere tontent—hewever, An sMAGES 


—tendeto}follew* two different paths, that of pipeline processing and 
that of storage, or frame buffer sepiinihiie iets processing is, 

as the name implies, a method where an operation is performed upon the 
scanned stream of picture elements as they are passed to the output 
display or recording unit, ina continuous fashion. The main advantage 
of this method is the high speed whth which operations may be performed, 
allowing the capacity for real-time motion within the £ atids, However, 
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[hereafter refered to ag) fixels, there igme capability for operations 


which require knowledge of, or comparisign with, past scene information. 
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Examples of operations that ma\ be performed with pipeline processing are 
color mapping, intensity remappink, window formation and digital keying. 
Frame buffer processing is the term applied to the 
storage of single frames, or sequences of frames; allowing transformations 
of pixel location within a frame, or over a number of frames. This method 
gives the capability of sialon operations which are time variant, 
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such as compression and expansion of the image, trajectory tracing, and 


pattern recognition. It's main disadvantage though, is the extremely 
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slow speed of storage and recall, yesulting from the sheer volume of 
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data involved. @est image processing systema Encorporate§ both methods, 


under computer control, to achieve a wid@€ variety of operations. 
The "Image Emulsifier" , which I have been helping the Vasulka Corp. 


to develop this semester, is just such a system. 


Description of the Image Emulsifier 





The Image Emulsifier system is a microprogrammable 
processor for the aquisition,storage,modification,and distribution of 
digitally encoded images of four bit resolution. The system currently 


consists of the following subsections: 


]) Quantizers- M&@\high-speed analog- to -digital converters bperating on 
Sere Arne 
separate buses] are employed for the real-time aquisition of ftw camera 


images by the systenf I constructed and tested the quantizers, which are | 


based around the(new) Advanced Micro Devices 6688 integrated circuit. 
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2) Image Buffers- Eight image buffers of 8k deep by four bits wide with 

high-speed access (200 nanosecond) time. Entering each buffer are 

four busses carring control signals,address and data information. At 

any time, any buffer may be connected to any one of the four busses. 

This allows for expansion of the image resolution by eoneseenation of 

several buffers onto the same bus; for example, connecting two buffers to 
a [the "a" | bus makes them appear to the bus as a widened, 8K by 8 bit buffer. 

Connection of a particular buffer to a bus is initiated by the buffer 

prtority logic, associated with each buffer. (i constructed four of the 


eight image buffers, and am in the process of testing them at this time.} 


3) Microprogrammed Microprocessor~ A high-speed (200 nanosecond) 
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microprocessor is connected to two of the four busses Ged, to)the image 





buffers. By controling two busses, reading or writing may be done to 

different locations in two different buffers. This gives the capability 

of variuous picture transformations, such as picture inversion, compression, 
ake ep 

expansion, edge extraction, and outlining.» A program control store 

fant ‘sequencer ‘gives instructions to the microprocessor, which is structured 

around an 80 bit pipeline. This control s¥ore has the capability for 

expansion,which will allow Prommed, or hardware stored programs to be 

permenently resident in the memory, to facilitate execution of the 

most commonly used microprograms. The microprocessor is based on the 
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the latest method of microprocessor design, that of the Bit Slice concept. | 





4) Microcomputer interface- The, particular interface involves connection 
of the Image Emulsifier to an LSI-11 microcomputer. The interface connects 
the LSI-11 to the microprocessor and to one bus of the buffers. A buffer 
request register is on the interface, to allow requests for usage of a 
particular buffer. te eecuied the data of a buffer, the LSI-1] must 
first request usage; if it is the highest priority device, a @rart 
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register will indicate that, iteifes usage of the buffers This grant 


register can be enabled to scnciage interrupts, upon getting the butfer 


















(that was} requested. Once a request/grant sequence is successful, a IK 


block of data may be written to, or read from, the granted buffer. To 


write to other sections of the buffer a memory page register is used, 


to map the 1K memory space into one of the pages of the 8K buffer. 
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The LSI-11 is also ed eli ea for setting the buffer priority 


registers on the eight image buffers, The LSI-11 must monitor the buffer 


activity, to time the change of epiorietes which affects the next memory 
cycle of the buffers. The LSI-11 is also responsible for loading the 


control store of the microprocessor, and has various control registers 
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5) Buffer and Screen Addressing circus try- iigemecttos)-aecomadates 


the generation of the horizontal and vertical timing to scan out, 
or write in, digitized video information to the image buffers. TRte-—witT 
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picture, shifts of the pasitétenm on the screen, and various pixel 
position remappings, 
imi inti =the phase of the high frequency 
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clock fto the video subcarrier phase, to avoid color shifts in the system. 


The timing of the X-Y timing chain , determining the pixel coorinates, is in 
turn locked to this high frequency clock. The high frequency clock itself 


sets the ‘time grid' which specifies the maximum and minimum resolution 
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attainable in the scanning of the puffers.| With a high frequency clock of 
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forms a total number of 512 picture elements, as the [mewimem) resolution G@®) 
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(Paemexssenr. J However, due to the buffer cycle time of 200 nanoseconds pike BuF Ee 
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resolution is further limited to 256 pixels faethennaximum | Thus,to store 
a 256 by 256 image requires some 65,536 memory locations, which is just the 
size of our complete buffer memory. By using all eight buffers simultainously 
one frame of 256 by 256 an ene em may be stored, but more frames may be 
stored simultainously if a lower resolution is used. For instance, if 
128 by 128 resolution is used. then 4 frames may be stored at one time, and 


if 64 by 64 resolution is used then 8 frames may be stored { MeontTsl 
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A mapping memory of 256 words by 8 bits is placed on the output 
of the X-Y address timing , one for the X and one for the Y addressing. 


These memories may be loaded with a function, which will translate the 
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sequential X and Y into F different) addressing scheme. These memories 


can be loaded during the inactive scan and utilized during the next active 


scan. 






“. The use of high speed multipliers on the X-Y signals is 
another saasipitey for image transformations. A 200 nanosecond 
multiplier of 16 bits width is employed on one bus to facilitate the 
loading of maps with transformations, or to use directly on the image 
data itself. In instances where transformations require pixels to be 
skipped or discarded, such as image compression, one place the task 
of which pixels to be skipped upon the round off error of the multiplier 
. _ 
itself. 
7 For more random addressing or data generation, a pseudo-random 
number generator has been implemented, with a variation on the shift 
register that allows the next input to be determined from the addition 
of selected bits of the register. This gives repeating sequences of 
programable length. However, by loading the register directly from the 
X-Y clock an unlocked noise source is then generated, due to the mismatch 
of the pseudo-random sequence to the video line/field blanking rate. 


eS 
the design and 





as 


wa 


construction of the a loop for the high frequency clock, 


and testing of the address coufiters and maps. / The-mubtiptier—circuit” 


6) Output Selection Logic- The output selection logic is responsible for 


eS 


routing the buffer outputs and other units to one or more of the four 


remains to be wired and tested, pos 





outputs of the system. A screen addressing unit further segments the 


outputs into four quadre@nts, according to the cartesian coordinate system. 








A sixteen section control register dictates the unit to be output in each 
quadrent of all four system outputs, allowing a single unit to be displayed 
on the entire screen or to segment the screenuinto B maximum off four areas, 


with a different unit being displayed in each area.[The-cutput_ueit-cam 
aiso—key,—or—switch point—by 





A use of these features might be the simultainous acquisition and display 
of four separate camera images, with a program running on the microprocessor 
that compares past and present frames of each camera to detect motion in 


front of any camera, blinking in red any area in which motion occurs. 


7) Digital -to-Analog reconversion- This section of the system will reconvert 


the four bit digital outputs into a lvp-p foammparsint <) video Sica ecumiaeecs 
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Based on the Analogic 8038 
6B (26 er) 


hybrid converter, the units &i11) feedg a standard color encoding unit, 





which accepts red,green,blue, and luminance signals to form a composite 


color video signal, pfxocerdeble—quelitys] 


[_ vin, conclusion, I think the work I was involved in this. 
past semester Sac Aimeheely rewarding, giving me invaluable experience 
in image processing, as wees high frequency digital circuits and 
techniques in computer interfacing ard control. This work was excellent 
background for my major in Electronic Arts, as aesthetic,as well as 


technical considerations ,all had to be weighted equally. 


